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ABSTRACT 



This document is a comparative report based on data that 
enables a different form of comparison to be made between Japanese and 
American eighth-graders. The data is drawn from assessments given in each 
country using tests developed via governmental agencies that were designed to 
mirror the basic curricula currently being offered to students in the two 
countries. This report compares the nature of these examinations, the 
expectations based on the curriculum, and student performance on the test 
items. This comparison, combined with an accompanying analysis of the 
curricular emphases, provides a picture of the differences in student 
performance that mark this study. Each of the studies examined content 
achievement assessments; students' and teachers' beliefs and attitudes; and 
environmental characteristics of the classroom, school, and community. The 
Japanese test items tend to present the mathematics assessed in forms that 
are more verbally intensive, more focused on strategies for problem solving, 
and more involved than the mathematics forms used in the National Assessment 
of Educational Progress. Sample test items, a content matrix for the U.S. 
test, and the framework for the Japanese test are included. Contains 29 
references. (DDR) 
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Chapter 1: Background and Purpose 

Introduction 

The definition of essential skills in mathematics has long been a goal of mathematics educators and 
others interested in the school mathematics curriculum. From the late 1970s forward, attempts have been 
made in the United States to provide a framework defining the basic essentials of mathematics that al I students 
should know and be able to apply (National Council of Supervisors of Mathematics, 1977; National Council 
of Teachers of Mathematics, 1980, 1981). Such recommendations for new directions in school mathematics 
called for a broader view of content and, in general, an increased emphasis on the development of student 
abilities to solve non-routine problems. These changes brought incremental changes in classrooms where 
the overall focus remained on number work and teacher dominated discussion of mathematics. In the late 
1980s, the National Council of Teachers of Mathematics (NCTM) produced its Curriculum and Evaluation 
Standards for School Mathematics (NCTM, 1 989). This document was designed to help strengthen the efforts 
in the United States toward developing foci for the mathematics curriculum. While its goals have yet to be 
implemented on a wide scale basis (Weiss, 1995), it has influenced changes made in the definition of 
mathematics used for the Department of Education’s National Assessment of Education Progress (NAEP) 
mathematics assessments (NAEP, 1987, 1988). 

Over this same period of time, from the 1970s through the 1980s, there was increasing evidence that 
the traditional mathematics curriculum was not preparing U.S. students to the same level of understanding 
as the Japanese curriculum was preparing students in Japanese schools (Husen, 1 967; McKnight, et al., 1 987; 
Travers & Westbury, 1989; Robitaille & Garden, 1989). Although the Second International Mathematics 
Study (SIMS) was unable to locate specific factors accounting for the differences in U.S. and Japanese 
students’ performance by the junior high school level, there was strong circumstantial evidence that the 
outmoded and unfocused mathematics curricular program of the early grades combined with the ways in 
which mathematics is taught in the elementary and middle grades was a probable cause (Leestma & Walberg, 
1992; Stevenson, et al., 1990; Stigler, et al, 1990: Stigler & Perry, 1988). Specific evidence indicated that 
students in Japanese classrooms had greater opportunities to learn and were often expected to master 
mathematical topics one- to two-years ahead of the time that American students first encountered similar 
topics. However, great emphasis was also placed on the differences in child-rearing and in the early 
educational experiences of children in the two societies as significant factors in creating the differences in 
mathematical performance. 

Studies comparing the mathematical achievement, or proficiency, of the two nations’ students have 
been hard to interpret. Studies by the International Association for the Evaluation of Educational Achieve- 
ment (IEA) have provided one vantage point. However, many critics contend that the IEA analyses are based 
on tests that contain items which not all students have had equal opportunities to learn in their schooling. 
The IEA tests are aimed to measure across a broad framework of mathematical skills and abilities, but 
invariably measure items included in neither country’s curriculum. Analyses indicated that the IEA tests 
were probably equally disadvantageous to each group of students. In the Second International Mathematics 
Study, the comparison groups of students were Japan’s seventh-graders and the United States’ eighth-graders. 
Studies carried out by Stevenson and Stigler and their associates measured student achievement with tests 
better fitted to the curricula studied by Japanese and American students, but their work only included students 
through the fifth grade level. In the early 1990s, new sets of data became available that made another form 
of comparison possible between Japanese and American eighth-graders. This was data from assessments 
given in each country, using tests developed via governmental agencies. In Japan, the test was developed by 
the Japanese National Institute of Educational Research and administered to a sample of eighth-graders in 
schools chosen according to some criteria. In the United States, the test administered to a random sample of 
eighth-graders was developed for the National Assessment of Educational Progress (NAEP). In both cases, 
these tests were developed to mirror the basic curricula currently being offered to students in the two 
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countries. Unlike other comparative studies, these tests were developed to assess the essential components 
of the taught curriculum in each country. During this period of time, both countries were undergoing 
significant changes in their school mathemaics programs (Ministry of Education, n.d.;NIER, 1992; National 
Council of Teachers of Mathematics, 1989; NAEP, 1988). 

In each country, students were assessed using examinations designed specifically for students in their 
country. The present paper compares the nature of these examinations, the expectations based on the 
curriculum, and student performances on the included items. This comparison, combined with an accompa- 
nying analysis of the curricular intents for students of grade two in the lower secondary school in Japan, the 
equivalent of grade eight in American schools, and students in grade eight in the United States provides a 
rich picture of the differences in student performance and curricular emphases and expectations that mark 
this study. From this point forward, we shall refer to both groups as eighth-graders. 

Data Sources 

The study makes use of data drawn from the Japanese National Institute of Educational Research’s 
Special Study on Essential Skills in Mathematics (NIER, 1992) and data from U.S. student performance on 
the 1990 and 1992 NAEP mathematics assessment (Mullis, etal., 1991, 1993). These data provide the first 
comparison of Japanese eighth-graders and American eighth-graders since the data collected during 1964 in 
the First International Mathematics Study (Husen, 1967). However, in this case, the performance of each 
group of students is measured on a different set of items developed in their own country. 

Format of the Studies 

Each of the studies described in the following sections examined content achievement assessments, 
students’ and teachers’ beliefs and attitudes, as well as environmental characteristics of the classroom, school, 
and community. While the full nature of the studies is briefly detailed to understand the context of the data 
collection, the following analyses are limited to the: 

• comparison of mathematics items tested on the Japanese NIER and American NAEP mathematics 
examinations, and 

• comparison of items to the espoused curriculum that each student was to have studied. 

In doing so, the NAEP item and ability classifications will be used to describe items of content and 
their intent (NAEP, 1988). 

Japanese Study. The Japanese data result from a study conducted by the National Institute of 
Educational Research to develop a definition of "essential achievement level" in mathematics and to 
determine the variables which may influence the formation of essential achievement. To accomplish this 
purpose, the study investigated the relationships between mathematics achievement and a number of 
variables, including: 

• Student attitudes: Motivation, interest, self assessment of performance or achievement, plans for the 
future, purpose of the studying, etc. 

• School environment: Teacher attitudes (teaching, teacher training, and specialization), school 
characteristics (number of teachers in each subject, number of students, number of classrooms, and 
frequency of use). 

• Environment outside school: Population of the town or city, distinction between urban and rural 
areas (NIER, 1992, p. 1). 
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This study encompassed the areas of mathematics, Japanese language, social studies, science, and 
English. The data examined in the present paper result from the 1990 testing of eighth-graders in mathematics. 
The actual assessment instruments were constructed by teachers from public schools, university faculty, and 
investigators from the ministry of education. 

The resulting examinations, to be described later, were administered to a nationally representative 
sample of students selected from schools in eight prefectures and two cities, see figure 1. The prefectures 
were Hokkaido, Iwate Ken, Chiba Ken, Toyama Ken, Shiga Ken, Okayama Ken, Kagawa Ken, and 
Kagoshima Ken. The cities were Tokyo and Nagoya. Four schools were selected in each of the cities and 
four schools were selected in each prefecture. In each prefecture, one school was selected from each of the 
following areas: 

• capital city of prefecture; 

• suburbs of each capital city; 

• city or town with population less than 100,000; and 

• areas neighboring city or town with population less than 1 00,000 (NIER, 1 992, p. 2). 

In each school selected, one eighth-grade classroom was selected and all students in that class served 
as subjects in the study for that school site. This resulted in 40 classrooms of students with 1,441 students 
for an average class size of 36 students. 

The administration of assessment instruments was done by the students 5 classroom teachers. For the 
eighth-grade mathematics assessment, there were two forms of the mathematics achievement instrument and 
a questionnaire for both students and teachers to acquire background information on student attitudes and 
environmental variables, as described above. These instruments were completed in two successive class 
periods. During the first session, students were given 45 minutes to complete the 22 mathematics achievement 
items on their tests. This gave Japanese students about twice the amount of time per mathematics item as 
given American students on the NAEP examination. During the following session, students were given 40 
minutes to complete the student background questionnaires concerning their attitudes and other relevant 
family and environmental information. This assessment took place in a time period between January 21,1 990 
and March 20, 1990. 

United States Study. The NAEP mathematics assessments have been given on a somewhat regular 
schedule since 1973, with assessments in 1973, 1978, 1982, 1986, 1990, and 1992. These assessments have 
measured the performance of U.S. students at both ages 9, 13, and 17, and grades 4, 8, and 12. Assessments 
are given in a wide variety of areas including mathematics, science, social studies, reading, and writing. The 
data collected are used to provide a rich mosaic picture of the current status of schooling in the United States. 
Achievement tests in the subject areas provide a picture of the proficiency levels of students at the respective 
ages or grades. They also provide information about long-term trends in what students know and are able to 
do in particular subject-matter areas in the curriculum. The assessments also collect background data on 
student attitudes and beliefs, as well as demographic information about their homes, families, and mathe- 
matical backgrounds. Additional information is gathered from teachers and administrators about the class- 
room and school environments. This data contains background information on teachers, their teaching styles, 
the school curricula, and the community. 

The NAEP mathematics assessment instruments are developed by a committee of mathematics teachers, 
university faculty, and mathematics professionals on the staff of Educational Testing Service, the contractor 
responsible to the government for conduct of the study. Prior to usage, the tests and questionnaires go through 
an approval process that includes representatives from both the policy and mathematics branches of state 
education agencies. 



Figure 1.— Map Location of Japanese Study Prefectures and Cities 
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The actual sampling of students and data collection for both the 1990 and 1992 NAEP eighth-grade 
mathematics assessments was conducted by Westat, Inc. The three-stage sampling design for NAEP 
assessments consists of the selection of geographical sampling units, the random selection of schools within 
that geographical region, and the random selection of eighth-graders from within the chosen schools. Some 
students, less than 6 percent, are excluded due to limited English proficiency or severe disabilities. In both 
1990 and 1992, 406 schools containing eighth-graders participated in the NAEP study. This number of 
schools represented an 84 percent participation rate in 1990 and an 87 percent participation rate in 1992 from 
among the schools randomly chosen to participate in the study. Within these schools, lists of students were 
again randomly sampled. In 1990, 8,888 students participated and in 1992, 9,432 students participated. In 
each year, the percentage of students participating represented 89 percent of the students randomly selected 
for participation. 

Each student received a booklet containing a set of questions about his/her mathematics background, 
three 1 5-minute blocks of mathematics items, and a set of questions about his/her motivation. Students were 
allowed 50 minutes to respond to the three blocks of mathematics items and the background information. 
The mathematics items in the three blocks to which an individual student would respond were portions of a 
broader set of mathematics items. This broader set of items consisted of 191 items in 10 different blocks in 
1990 and 235 items in 16 blocks in 1992. These items were spiraled into students’ test booklets according 
to a balanced incomplete block design which allows for broader coverage of the mathematics content while 
minimizing the burden placed on any individual student. Thus, during the 45 minute mathematics examina- 
tion period, an individual eighth-grader in the U S. had approximately 57 items to respond to in 1990 and 44 
items to respond to in 1 992. This decrease in the average number of items per student between the assessments 
was accompanied by an increase in the number of items requiring students to construct their own response 
to the item, rather than select a response from a multiple choice question. In these years, the individual 
eighth-grader in the U.S. had approximately 2 to 2.5 times as many items as Japanese students were asked 
to do in a similar time period. 

Trained field staff provided by Westat collected all of the data for the 1992 mathematics assessment. 
The data was collected from January, 1992 through mid-March, 1992. The materials collected were scored 
by National Computer Systems. This scoring included the open-ended student constructed response items 
included as part of the assessment. (Mullis, et al., 1991, 1 993). 
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Chapter 2: The Mathematics Assessments 

The mathematics content tests given students as part of both nations’ examinations were intended to 
be reflective of content that was deemed appropriate for eighth-grade students in Japan. In the United States, 
items were chosen according to the content matrix established for the 1990 and 1992 NAEP mathematics 
assessments. This matrix was developed in late fall 1988 by a committee of teachers of mathematics, state 
supervisors of mathematics, and university faculty in mathematics education and mathematics. In doing so, 
the NAEP mathematics framework committee, chosen by the Council of Chief State School Officers, used 
a draft of the NCTM Standards as a guide to item specification and grade level emphasis. A copy of this 
content matrix is included in appendix A. This framework divides school mathematics into five major content 
areas: Numbers and Operations; Measurement; Geometry; Data Analysis, Statistics, and Probability; and 
Algebra and Functions. The content to be included in each of these areas is rather self-explanatory. A more 
detailed definition of each of the areas is given in appendix A. For each subarea of content within the five 
groups, a darkened bullet is displayed to the right in the appendix materials indicating if that material is 
appropriate for assessment purposes at grade 4, 8, or 12. These judgments were developed by the NAEP 
mathematics framework committee and were open to comment by mathematics educators nationwide before 
final revisions and final adoption. 

The Japanese mathematics assessment was constructed based on a framework developed specifically 
for the research study of which it was a part (NIER, 1992). The material accompanying the assessment 
materials details attempts to develop a position on essential mathematics achievement. In attempting to move 
to such a definition, the researchers established the following tenets: 

• Mathematics achievement extends beyond the acquisition of concepts and related skills, to include 
individuals’ attitudes and motivations in mathematical situations. 

• While mathematics problems worked in class often have little relationship to reality, individuals 
should be able to successfully deal with mathematics problems in their everyday lives and the 
classroom, by solving them and verifying their solutions. 

• Individuals should be able to productively transfer their mathematical concepts and skills, as well 
as problem-solving abilities, from mathematics classrooms to other situations in their lives. 

Using these tenets, the researchers at the Japanese National Institute of Educational Research (NIER, 
1992) defined essential achievement in mathematics as: 

Essential mathematical achievement is a necessary level of achievement in order to solve 
problems mathematically in academic institutions and society. It includes the possibility of 
being able to cope with new situations. From the viewpoint of the development of students, 
the contents of the basic achievement level ought to make it possible for teachers to guide 
and teach students. The essential achievement level should be learned effectively in the 
school system (p. 1 1). 

In many ways, this movement to define essential achievement in mathematics for Japanese eighth-grad- 
ers is similar to the attempts to define achievement levels for American students of the same age for the 
NAEP assessment analyses. 

In structuring the mathematics assessments, the Japanese developers created a three-dimensional matrix 
framework. The dimensions of the matrix were behavior types, contents of mathematics, and mathematical 
processes. The subcategories of each are similar in some ways to the contents of the NAEP framework. The 
behavior types dimension was further divided into the behaviors of knowledge, understanding, thoughts, 
skills, and attitudes. The contents of mathematics dimension was divided into three branches: numerical 
content, geometrical content, and relational content. The third dimension, mathematical processes, was 
broken into the subcategories of mathematicalization, mathematical transaction, and mathematical verifica- 
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tion. These three process categories define the dynamic use of mathematics by students as they apply it to 
translating problems into mathematics, using mathematical knowledge to solve the problems, and verifying 
that the results obtained fit the problem and answer the original questions. A further description of the 
Japanese framework is contained in appendix C. 

Japanese Content Tests 

The 40 items chosen by the research group designing the assessments for the Japanese study were 
placed into two forms, C and D, for ease of administration. Each form contained 22 items. Four items were 
included on both forms of the examination as a cross-check on the comparability of the two forms. Thus 
each form had 18 unique items and the set of four items used as a cross-check between Forms C and D. 
Students in each of the classes were divided into two groups and the members of each group received one 
of the forms of the test. They were given 45 minutes to complete the 22 items on the form. All, but two, of 
the items were multiple-choice in format. The two exceptions were an item which required students to sketch 
the planar net associated with a spatial perception item and another item which required students to draw a 
line having a special property onto a figure. 

The items included in the 40-item pool reflected current thinking about the nature of mathematical 
essentials all students at the grade level should have. However, eight items were chosen from earlier studies: 
the survey of achievement level ( 1 98 1-82) and the academic achievement survey which was conducted before 
1981 (NIER, 1992). 

Using the Japanese mathematics framework described above and in appendix C, the 40 items were 
categorized as shown in table 1. 



Table 1. — Categorization and Percent of Items on the Japanese Test Using 
the Japanese Essential Mathematics Framework 



Mathematical 


n* 


Mathematical 


n 


Mathematical 


n 


Behavior Type 




Content Type 




Process Type 




Skills 


9 










Knowledge 


14 


Numerical 


15 


Mathematicalization 


8 


Understanding 


15 


Geometrical 


14 


Mathematical Transaction 


32 


Thoughts 


11 


Relational 


11 


Mathematical Verification 


0 



*Some items are categorized in more than one behavior type category. 

An analysis of the items reveals that many skills were required in order for the students to correctly 
respond to some of the items measuring knowledge, understanding, and thoughts. The behavior type called 
"Thoughts" dealt with students ability to interpret new problem types using previously known facts, concepts, 
and principles. In the Mathematical Process Type dimension, no item was categorized as measuring the 
process of verification. 

The 40 items on the Japanese examination were also categorized using the 1992 NAEP framework. 
This categorization was done independently by two mathematics education researchers at the NAEP offices 
at Educational Testing Service and the author. There was a 95 percent agreement on the categorization of 
items into the five major content categories and a 90 percent agreement on the categorization of the items 
according to mathematical ability categories. 
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The 40 items spread across the NAEP categories is illustrated in figure 2. One can see the heavy emphasis 
in the examination on items measuring geometry and algebra and functions content. This “spiked” emphasis 
in two content areas with a relatively low number of items in the other content areas is a curricular emphasis 
pattern similar to that noted in the SIMS study. Some have argued that a targeted emphasis within a year on 
fewer topics will potentially lead to greater growth than a broader smorgasbord of topics with smaller 
expectations of growth (McKnight, et al., 1987). A content emphasis comparison of Forms C and D showed 
that the two forms were well balanced in the distribution of items from the five NAEP content areas. 




Numbers Measurement Geometry Probability Algebra 

and and and 

Operations Statistics Functions 

Content Area 



NAEP Content Tests 

The NAEP content tests for 1990 and 1992 were developed in accordance with the framework and item 
distributional requirements set out by the design developed in 1988 (NAEP, 1988). This resulted in a set of 
191 items distributed across 10 test blocks in 1990 and 235 items distributed across 16 test blocks in 1992. 
These blocks were then reassembled into test booklets of three blocks each. 

In 1990, grade eight students were allowed to use calculators on two of these blocks and a ruler and 
protractor on another block of items. In 1992, eighth- grade students were allowed to use calculators on three 
of the blocks and a ruler and protractor on one other block. For another block in 1992, students were given a 
packet of geometric shapes to use in responding to a number of items. In both years, three blocks of items 
were administered through the use of a paced-audio-tape format to move students through a series of items 
dealing with estimation, data analysis, and algebra. The use of the paced-tape approach was chosen to forestall 
computations on the estimation items and to ease the reading requirements while assisting in time management 
on more complex items. 

In 1990, 149 of the 191 items at grade 8 were multiple choice items (47 from the paced-tape blocks) 
and 42 items required students to construct a computed answer or short response to an item. In 1992, the 235 
items at grade 8 included 164 multiple choice items (46 from the paced-tape section), 65 items requiring 
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students to construct a computed answer or write a short response to the item, and 6 items that required an 
extended written response on the part of the student. 

Independent of which set of three blocks an individual student received in an individual student test 
booklet, they had 45 minutes to respond to approximately 57 questions in 1990 and 44 questions in 1992. 
This was a much higher level of average number of items per student for a comparable time period than was 
observed in the Japanese tests. This placed from 2.5 to 2 times as much item response load per time period 
on individual U.S. students, plus 22 percent of the NAEP items in 1990 and 30 percent in 1992 required U.S. 
students to produce some form of self-constructed written response. 

Figure 3 illustrates the balance of items across the five content categories for both the 1990 and 1992 
NAEP mathematics assessments. The allocation of items to categories exhibits a more rectangular distribution 
of items across the five content categories than was observed for the Japanese tests. No “spikes” were 
observed, unless one would say that the Number and Operations category is perhaps a “spike.” 



Figure 3. — Percent of Items by NAEP Content Areas for the 1990 and 1992 
NAEP Tests 

r 30 




Numbers Measurement Geometry Probability Algebra 

and and and 

Operations Statistics Functions 



Content Area 

Figure 4 provides a direct comparison of the allocation of items to the five NAEP content categories 
across the two examinations. Here one sees immediately the pattern of a “spiked” approach which gives a 
heavy emphasis to a few areas in deference to others. The Japanese model provides heavy coverage of student 
achievement in the areas of geometry and algebra, but provides little coverage for the other three areas. This 
undoubtedly signals what is important to the students and teachers when the examinations come from the 
ministry offices. In alike manner, the U.S. examinations may be signaling that numbers and operations should 
be the main focus of our curriculum for eighth-graders. This appeared to be one of the findings of the SIMS 
study (McKnight, et al, 1987). 
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Figure 4.— Comparison of the Distribution of Items to Content Areas in the 
Japanese Essential Skills and the U.S. NAEF Examination 
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Figure 5 shows the distribution of the items to the three NAEP mathematical abilities categories described 
in appendix B. These data indicate that the Japanese may have placed somewhat more emphasis on conceptual 
understanding and slightly less on straight problem solving abilities in designing their assessment items. 

Figure 5.— Percentage Distribution of Items on the NAEP and Japanese Tests to 
the Mathematical Abilities Categories 
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Chapter 3: The Items 

In the following sections, each devoted to a content area, examples of the items from each assessment 
will be provided to compare the nature of the items and the cognitive load each presents for the students 
taking the examinations. In a direct comparison of items from the NAEP pool, 38 items were classified in 
the same cells as those from the Japanese assessment. Of these, 21 are still in the secure pool for use in later 
NAEP assessments for the determination of proficiency trends in the content areas. These items will not be 
displayed, but performance on those from the grade eight examinations and general descriptions of their 
content and format will be provided to give some idea of the comparability of the assessments and their results. 
More in-depth analysis of the 1 990 and 1 992 NAEP assessments and items can be found in the national reports 
describing these assessments' (Mullis, et al., 1991,1993). 

Numbers and Operations 

There were two items categorized as Numbers and Operations items on the Japanese assessment. These 
items were number C16 (designator indicates form and item number) and D2. These items are illustrated 
below, along with a strip indicating the percentage of students responding to each alternative response. The 
asterisk indicates correct response. 



C16 In die figure below, two numbers are connected by an arrow. For example, 70 — ^ 14 

indicates that 70 can be divided by 1 4. When you can use any numbers in ( ) and □ , 

except those already used in the figure, what number will fit into ( )? Select one answer 
from (1) to (5). 



(A) 3 

(B) 5 

(C) 7 

(D) 10 

(E) 15 




1) 1.9 2) *60.7 3) 20.7 4) 12.2 5) 3.0 (no answer) 1.4 

Item C16 required students to deal with the relation of "is divided by" as signified by the arrow. Students 
had additional needed information presented in the form of a directed graph. The number theory required in 
the item is contained in the middle school mathematics programs at most U. S. schools. However, few U. S. 
children would have seen the directed graph representation for the "is divided by" relation. Such graph theory 
content is only now beginning to enter the secondary school curriculum under the guise of "discrete 
mathematics." 



There was one item on the 1990 grade eight NAEP examination corresponding to item Cl 6. Item 15 
shown below was administered to both grade 8 and grade 12 students in 1990. It measures the related number 
theory concept of least common multiple. The fact tha^ the third number is unknown makes this problem 
slightly more difficult than item Cl 6. Eighteen percent of the grade 8 students and 30 percent of the grade 
12 students correctly answered this item in 1990. 



NAEP 1 5. The least common multiple of 8, 1 2, and a third number is 1 20. Which of the following 
could be the third number? 

(A) 15 

(B) 16 

(C) 24 

(D) 32 

(E) 48 

A) *18 B) 24 C)39 D)8 E) 10 




Item D2 of the Japanese examination required students to compare five numerical expressions where 
operations involving signed numbers were performed with an unknown number a and to decide which of the 
five expressions represented the "biggest number." While not required, no qualifications were placed on the 
values that a itself could be. 

Like its predecessor, Item D2 had no direct counterpart on the 1992 NAEP examination. The content 
of the item was judged as appropriate for assessing at the grade eight level, but no specific item measuring 
the same content was included on the 1992 assessment. 



D2 Which one is the biggest number among the following? Select one answer from (1) 

to (5). 

(1) a - (-2) 

(2) a -(-1/3) 

(3) a-0 

(4) a + (-1/3) 

(5) a + (-2) 

1) *86.5 2) 2.9 3) 5.2 4) 2.2 5) 2.9 (no answer) 0.1 



Measurement 

There was only one Measurement item included on the Japanese essential mathematics assessment. 
This was item Cl 1 . This item dealt with the concept of surface area. The presentation of the item included 
a graphic that could be employed by the student taking the examination to recall what "surface area" refers 
to if they had forgotten. Students then had to search through at least three of the possible responses before 
they could identify the correct answer. This item was difficult for the Japanese students as only 36 percent 
of the students answered it correctly. 



Cl 1 . There are 7 cubic blocks with a 

side length of 1 cm. Consider the 
surface area as we make a solid 
by using all blocks. For example, 
the surface area of the figure at 
the right is 30 square cm. In the 
following 5 solids, there is one 
solid which has a surface area 
different from the others. Select 
one answer from (1) to (5). 





LZvy 













( 1 ) 






1) * 36.3 2) 15.1 3) 14.8 4) 25.3 5) 6.6 (no answer) 1 .9 
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There were two unreleased (secure) items on the 1990 NAEP that correspond to item C 1 1 . The first of 
these required students to identify which of five possible alternatives gave the surface area for a rectangular 
solid of blocks when provided with a graphic illustration of a pile of blocks similar to that shown in Cl 1. 
Only 20 percent of the students taking this item completed it correctly. The cognitive load for this item was 
less, since there were no indentations in the surface. The derivation of the answer was very straightforward. 
There was little a student could do with the item stem to discover the meaning of "surface area" had they 
forgotten, as was possible in item C 1 1 . 

The other corresponding item on the 1990 NAEP examination was an item which presented students 
with a planar net for a cube and asked them to identify a pair of opposite surfaces. Performance was 
considerably higher on this item, with 51 percent of the students correctly answering the item. This item is 
considered to correspond to Cl 1 because some spatial representation skills are needed to correctly complete 
either item. 

Geometry 

There were 13 geometry items in the pool of items com prising the Japanese examinations. The following 
analysis treats them in the order that they appear in the NAEP content framework. Item C21 presents a pair 
of figures showing a triangular prism being unfolded and students are asked to use a compass and ruler to 
complete a representation of the fully unfolded prism. Although 41 percent of Japanese eighth-graders 
correctly answered this item, little information was given as to how a response was coded as correct with 
regard to the use of the ruler and compass representations of the student’s response. This was one of the two 
items that were not multiple choice items. 

C21. There is a triangular solid shown below on the left. We draw the figure unfolded 

halfway as shown below on the right. Draw a completely unfolded figure using a 
compass and a ruler. 





3) 22.3 4)1.8 



5)6.0 (6) 9.8 (no answer) 10.0 



The 1 990 NAEP contained a counterpart item to C2 1 . This secure item asked students to imagine cutting 
a three dimensional paper shape along an indicated dotted line. They were then asked to write a short 
description of what the shape of the "flattened" paper would be. Seventy-eight percent of U.S. eighth-graders 
were able to correctly respond to this item. 



Item C14 asked students to give the size of an angle formed when two triangle rulers are placed atop 
one another in a manner shown in the accompanying illustration. Students had to assume that one of the 
drafting triangles was a 45°- 45°-90° triangle and the other was a 30°- 60°- 90° triangle. There are a number 
of ways of solving the problem posed. One could use the angle sum theorem for a triangle and either measures 
of exterior or vertical angles. Those with good estimation skills could have recognized that the angle formed 
was greater than 90° and less than 120° and have selected the correct answer on the basis of that information. 
Sixty-six percent of the Japanese eighth-graders answered the item correctly. 

19 
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C 1 4. In the figure shown below, when we put a pair of triangular rulers atop of one another, 

what is the degree measure of angle al Select one answer from (1) to (5). 

(1) 60 degree 

(2) 75 degree 

(3) 90 degree 

(4) 105 degree 

(5) 120 degree 



1) 1.6 2) 8.1 3)10.3 4) * 65.9 5) 13.0 (no answer) 1 .0 




The 1992 NAEP contained a secure item that is somewhat similar to Cl 4. This item asked students to 
find the measure of an interior angle of a triangle given the measures of another interior angle and a remote 
exterior angle. Hence, the exterior angle theorem could be applied. Only 31 percent of U. S. students were 
able to correctly answer this multiple choice item. The correct answer to this item was not easily estimated 
from visual inspection. 

Item D12 asked Japanese students to identify which of a number of angles has to have the same measure 
as^o so that lines / and m will be parallel. This item asks students to recall some geometric principles 
concerning parallel lines and the measures of corresponding and vertical angles to correctly answer the 
question. The data show that 70 percent of Japanese eighth-graders were able to successful ly select the correct 
answer. There was no directly comparable NAEP item on the 1992 assessment for item D12. 



D12. 



In the figure shown, a line k intersects two lines / and m. Which angle has to be equal 
to L a in order for the line / and line m to be parallel? 




1) 0.7 2)13.2 3) 9.1 4) *70.3 5) 6.6 (no answer) 0. 1 



Item D 1 4 presented Japanese students with another geometric situation intending to require knowledge 
of polygons, their sides, congruent triangles, and related angles and their measures for a correct solution. 
Forty-five percent of Japanese students were able to respond correctly to the item. 



D14. 



In the hexagon ABCDEF at the right, AB \ \ DE, AB=DE, BC=FA, CD=EF. 
In this case, which one is correct? Select one answer from (1) to (5). 



(1) L BCD =E AFE 

(2) LCBA=LFED 

(3) BC\\EF 

(4) CD 1 1 FA 

(5) BE and CF intersect 
each other at the midpoint. 




1) *44.6 2) 27.6 3) 4.5 4) 2.1 5) 20.p g (no answer) 0.6 
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Like the preceding item, there was no direct counterpart to this item on either the 1990 or 1992 NAEP 
examinations. This item called for several applications of geometric properties usually taught in either the 
ninth- or tenth-grade in the U. S. It seems, based on students’ opportunity to learn, that such an item would 
be inappropriate at the present for the great majority of U.S. eighth-graders. Visually the item is much easier 
than actually having to construct the location of point P. 

Item D 1 5 asked Japanese students to select one of five possible locations for a point P on the hypotenuse 
of aright triangle ABC so that the two resulting triangles ABP and BCP would be similar. Students familiar 
with the family of theorems about the altitude to the hypotenuse of a right triangle would be able to answer 
the question quite rapidly. Many students may have seen this content in a verification of the Pythagorean 
theorem. Other students, familiar with similar triangle relations, could solve the problem working from basic 
principles. In any case, 58 percent of Japanese students correctly answered the item. 



D15. 



There is a right triangle as it is shown at right. 
Take point P on the hypotenuse A C and connect it 
with the vertex B. In order for &ABP and A BCP 
to be similar, where do you place point PI Select 
one answer from (1) to (5). 



A 




(1) AP=BC 



(2) AP=PC (3)lAPB=\00° 



(4) L APB=90° (5)&ABPand 

&BCP can be 
never similar 






The only corresponding item on the 1990 NAEP assessment for eighth-grade students was a secure 
item asking students to find the length of one side of a triangle given that the triangle is similar to another 
triangle. All of the necessary measures of sides are given and the two triangles given are positionally arranged 
so that corresponding sides are easy to note in setting up the required correspondence of sides and their 
measures. While considerably easier to answer, only 48 percent of the U.S. sample correctly responded to 
this item dealing with similar triangles. 

Item Cl 8 asked students to imagine a dynamic situation as one vertex of a triangle moves along a line 
parallel to the opposite side of the triangle. Students are asked to identity which of a group of alternatives 
captures all of the possible changing measures associated with the changing triangle APD. The important 
thing for students to note is that the area of triangle APD does not change. Sixty-eight percent of Japanese 
students were able to identify that the length of segment BP, the length of segment PC, the area of triangle 
ABP, and the area of triangle APD would all change under the movement shown. 
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C18. 



Rectangle ABCD is shown below. A point P moves from the vertex B to the vertex C on 
the side BC with a speed of 1 cm per second. When we connect the point P, the vertex A , 
and the vertex D, which of the following answers includes all of the parts of the diagram 
that change as point P moves? Select one answer from (1) to (5). 



■ (A) The length of BC (B) The length of BP 

1 (C) The length of PC (D) The area of PEABP 

| (E) The area of A APD (F) The area of A PCD 




1) 6.9 2) 6.6 3) 5.8 4) 9.9 5) *68.3 (no answer) 2.6 



The only item in the pool ofNAEP items resembling the content contained in Cl 8 was an item given 
to 1 2th-graders in the 1 992 NAEP. This slightly more difficult item, illustrated below, asks students to explain 
why three triangles having one side as a side of the same rectangle and their third vertices located on the 
opposite side of the rectangle all have the same area. This item calls on content they should be able to discern 
from the drawing, that is, they all have the same base and altitude, hence the same area. The resulting 
performance, 10 percent answering correct, shows that U.S. grade 12 students were not able to extrapolate 
from the drawing and their knowledge of the area formula for a triangle to develop a correct explanation. 
This is consistent with past NAEP findings about student knowledge of measurement formulas. 

NAEP 18. In the rectangle ABCD below, explain why triangles AED, AFD, and AGD have equal 
areas. 




Item CIO asked Japanese students to select which approach to a problem might yield a desired result. 
To establish the desired relationship, the student must establish that the segment A4 1 is perpendicular to the 
base of the prism. To do this, segment AA' must be perpendicular to two intersecting segments intersecting 
at A in the base ABCDE. As the data indicate, 44 percent of Japanese students answered correctly. Like the 
previous items, this question requires students to have a considerable command of basic relations from 
elementary plane and solid geometry. 




18 



22 



CIO. 



In the figure to the right, we want to find 
whether or not the edge AA' is vertical 
to the base ABCDE. How can we find 
out? Select one answer from (1) to (5). 




( 1 ) find whether the edge AA ' 

(2) find whether the edge AA' 

(3) find whether the edge AA' 

(4) find whether the edge A A 1 

(5) none of the above. 



and the edge AB are vertical. 

and the edge AB are parallel. 

and the 2 edges AB and AE are vertical. 

and the line segment AD are vertical. 



1) 17.4 



2)9.8 



3) * 43.8 4)15.6 5)12.1 (no answer) 1 .4 



While there was no direct companion item for CIO on the eighth-grade portion of the 1990 NAEP 
examination, there was one item on the grade 12 1990 NAEP examination that paralleled some of the ideas 
of CIO. This secure item asked students to select the shape of an intersection of a plane with an illustrated 
right pentagonal prism. Students needed to visualize the intersection, which includes points on both bases of 
the prism, to answer the question. Twenty-seven percent of U. S. grade 12 students were able to correctly 
answer this item. This item differed from the previous item on the Japanese examination in that students had 
to visualize the solution and then select the choice describing it, rather than recall a definition of what 
perpendicularity means for a line and plane in space. 

Item Cl 2 presented Japanese students with two examples of the equal additions property embedded in 
the angle addition and area addition properties of elementary plane geometry. The equal additions property 
was stated in both geometric representations and then students were asked to deal with the decoding of the 
property into its algebraic representation. As the data show, 45 percent of the students were able to make this 
transition correctly. 



Cl 2. In the figure below, there are two characteristics. 

WhenZ. ABE =L DBC, WhenA ABC = &DEF, 

thenZ ABD -L EBC. then the area of the diagonal lines (///) 

= area of the dots (: : : :). 





We can prove these 2 characteristics based on the same rule. Find the rule from (1 ) to (5). 

(1) When a = b, a + c = b + c (a, b, c are positive value) 

(2) When a = b, a - c = b - c (a, b, c are positive value) 

(3) When a = b, a x c = b x c (a, b, c are positive value) 

(4) When a = b, ale = b/c (a, b, c are positive value) 

(5) When a = b and b = c, a = c (a, b, c are positive value) 

1) 12.9 2) *44.6 3)11.0 4) 8.2 5) 20.7 (no answer) 2.5 
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One item on the 1992 NAEP which dealt with the angle addition property in a different way is shown 
below. This item presented eighth-grade students with two complementary angles and the information that 
they each were bisected. Students were asked to give the measure of the angle formed by the two bisecting 
rays. NAEP data show that 23 percent of U. S. students were able to select the correct answer to the item. 
These items differ in that the Japanese item is more symbolic and the NAEP item depends more on a visual 
interpretation. 



NAEP13. The sum of the measures of angles 1 and 2 in the figure below is 90°. What is the 
measure of the angle formed by the bisectors of these two angles? 



(A) 60° 
~ } 45° 
30° 



(B 

(C 



(D) 20 

(E) 15 c 



A) 21 



B) *23 C) 27 




Item DIO asked students to describe the number of points in the plane that satisfy the locus of being 
one centimeter from both lines / and m as shown in the accompanying illustration. Only 36 percent of Japanese 
students were able to correctly respond to this item. Locus problems are not a part of the standard curriculum 
at this level of education in Japan. 



DIO. 



In the figure below, two lines / and m intersect each other. How many points 
are there that are 1 cm distant from both line / and line m ? Select one answer 
from (1) to (5). 




1 point 

2 points 

3 points 

4 points 
infinite points 



1) 7.9 2) 9.8 3) 3.1 




On the 1992 U. S. examination there was only one item that directly dealt with a locus problem. This 
was one of the extended student constructed response items. These items required students to spend a 
minimum of five minutes working on the item and then to write a paragraph, along with accompanying 
illustrations, to fully respond to the requirements of the problem. This item was graded according to a scoring 
rubric that allowed for students to be judged along a five-point scale with major levels of incorrect, minimal, 
partial, satisfactory, and extended responses. The percentage of students reaching these levels of response 
were 45, 22, 13, 4, and 1 respectively. Thus, only 18 percent of the students received a score of partial or 
better, 5 percent received a score of satisfactory or better. Sixteen percent of the students did not even respond 
to the item (Dossey, et al., 1993). 
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NAEP27. 



Radio station KMAT in Math City is 200 miles from radio station KGEO in Geometry 
City. Highway 7, a straight road, connects the two cities. 

^Ttoricasts can , be rece i ve d up to 1 50 miles in all directions from the station and 
KGEO broadcasts can be received up to 125 miles in all directions. Radio waves travel 
rrom each radio station through the air, as represented below. 



Radio 

Station 




■Radio 

Wave 



On the next page, draw a diagram that shows the following. 

• Highway 7 

• The location of the two radio stations 

• The part of Highway 7 where both radio stations can be received 



Be sure to label the distances along the highway and the length 
highway where both stations can oe received. 



in miles of the part of the 



Item C 1 7 involved Japanese students in determining the validity of an assertion. In this case, the students 
were involved in considering a proposition about the length of a circuit formed by connecting, in order, the 
midpoints of the sides ofa closed convex polygon. They are introduced to a conjecture made by a hypothetical 
student Taro and asked whether Taro is correct or not. Some of the choices allow the student to further qualify 
their answers. The results show that 46 percent of the students were able to correctly complete the item. 



C17. 



figures (a a ) P a P nd b (b) aUSe ^ discovered the followin g characteristics when he saw two 



“In both cases, circumferences of the 
midpoints of each side are one half of the 



figures which are created by connecting 
circumferences of original figure.” 





Is Taro right? Select one answer from (1) to (5). 

(1) Right 

(2) (a) is right, but (b) is wrong 

(3) (a) is wrong, but (b) is right 

(4) both (a) ana (b) are wrong 

(5) None of the above 

23-1 2) *45.9 3) 15.0 4) 10.7 5) 4.0 (no answer) 1.4 
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There were no comparable geometry items for U. S. grade eight students included in the 1992 NAEP 
assessment, as verification of geometric conjectures receives low priority in the current mathematics 
curriculum at the eighth-grade level. 

Item D13, like the previous item CIO, asked students to make rather extensive use of their knowledge 
of geometric modes of argument to answer the question proposed. Here students had to read through a 
problem outlining a given statement to prove. Then they had to consider Hanako’s reasoning presented in 
the stem of the problem. Based on the flow of the argument in the reasoning, the students then had to select 
which pair of reasons best fit into the flow of the argument being formed by Hanako in response to the 
problem presented. This problem samples students’ understanding of the nature of a mathematical argument, 
as well as their knowledge of geometrical principles. Students are required to bring together an assumption 
and complete a side — side — side triangle congruence argument. The form of question is very different from 
ones used as part of the NAEP mathematics assessment. Data from eighth-grade students in Japan indicate 
that 52 percent of them were able to correctly answer the item as presented. 

D13. In order to prove the following problem, Hanako solved it in the following way. 



Problem 

In the figure at the right, D is the midpoint 
of base BC of isosceles triangle 
ABC. Prove that a line segment AD divides 
vertex angle A into one naif. 




Hanako' s way of thinking 

In order to prove L BAD = L CAD, I should prove that A ABD and A ACD 
are congruent. I examined corresponding siaes and angles of A ABD and A ACD. 
Since &ABC is an isosceles triangle, AB = AC. Also, AD is common to 
both triangles. Moreover, from the assumption, ( a ). 

Therefore, from ( b ), A ABD and A ACD are congruent. 



In Hanako's way of thinking, what equation and sentence should be in ( a ) and ( b )? 
Select one answer from (1) to (5). 



(1) fa) LBAD= LCAD 

(b)Two pairs of corresponding sides are equal and the angles that are in 
between these two lines are equal. 



(2) fa) L ADB = L ADC 

(b) A pair of corresponding sides is equal and the angles which are located at 
Doth edges of the sides are equal. 

(3) (a) LADB = LADC 

(b)Two pairs of corresponding sides are equal and the angles in between 
these two sides are equal. 



(4) fa) BD = CD 

(b) A pair of corresponding sides is equal and the angles which are located at 
Doth edges of the sides are equal. 

(5) fa) BD = CD 

(b)Three pairs of corresponding sides are equal. 



1) 25.2 2) 6.2 3) 9.1 4) 7.6 5) *51.6 (no answer) 0.3 
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There are no comparable items for eighth-grade students in the U. S. on the 1992 NAEP assessment. 
Such forms of verification are not a major portion of the grade eight geometry strand in most schools and 
most students do not have an opportunity to study the various methods for proving a pair of triangles 
congruent. 

The Japanese item D22 asks students to consider a given rectangle, representing a tract with a circular 
pond included in the tract. Students are asked to draw a line on the figure through the center of the circle that 
will bisect the shaded region in the interior of the rectangle. This item calls on student understanding of 
symmetry and its relationship to area, as well as to their overall knowledge of area as a concept. The data 
indicate that 65 percent of the students were able to correctly indicate a line running through points O and E 
as the correct solution. This item was the second item that was not given in multiple choice format. No 
information was presented on the rubric used to assess student responses or what the percents associated with 
the other response values represented. 

D22. In the figure at the right, within 

rectangle ABCD (the intersection of 
diagonal lines is E), is a circular pond 
with a radius of 5 m (the center is 

0) . Draw a line which goes through the 
center 0 of the circle and bisects the 
area of the rest of the figure (shaded 
area). 

1) *64.7 2) 0.6 3) 7.6 4) 2.5 




There was one secure item on the 1992 NAEP that had some similar properties to item D22. This item 
was one of the secure items from the 1992 assessment that made use of the geometric pieces that students 
were provided for one of the blocks of items. Students were given the statement that one student said that 
the two pieces had the same area, another student disagreed. The student taking the test then had to take a 
position backing one of the students and explain why that particular position was correct. They were explicitly 
asked to "use words or pictures (or both) to explain why." Only 22 percent of the U. S. eighth-graders could 
successfully complete this task, which required a much easier dissection than the corresponding Japanese 
item D22. 



Item D1 1 is another question dealing with solid geometry. In this item, students are presented with a 
planar net for a square pyramid and asked to indicate the number of ways that a fourth triangular side could 
be added to the net so that, when folded, the four triangles and base will come together to form a pyramid. 
The data indicate that 52 percent of the Japanese students were able to correctly determ ine that there are three 
different locations where the fourth triangle could be correctly added to complete the desired figure. 



Dll. 




We can make an unfolded figure of a 
square pyramid if we add one more 
triangle in the figure at the right. How 
many wavs of adding a triangle are 
there? Select one answer from (1) to 

( 5 ). 

(1) 1 way 

(2) 2 ways 

(3) 3 ways 

(4) 4 ways 

(5) 5 ways 




1) 9.8 



2) 20.6 3) *51.5 4) 13.2 
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5) 4.1 
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(no answer) 0.8 



An analysis oftheNAEP items from 1990 revealed one secure item that was somewhat similar to D1 1. 
This item required students to identify which, of a number of potential nets, that when folded, would not 
form a particular geometric solid. This item lacked the combinatorial aspect of asking how many ways an 
additional needed piece could be added to a partially completed net, but required that more information be 
processed. Fifty-eight percent of U. S. eighth-graders were able to answer this easier item. Analysis of 
upcoming NAEP assessment blocks for mathematics indicates that another similar item will, most probably, 
be included in the next mathematics assessment. 

Data Analysis, Statistics, and Probability 

The Japanese essential mathematics assessment contained three items measuring content that would 
be classified in the NAEP content category of Data Analysis, Statistics, and Probability. This content is an 
area of some interest in the curriculum for Japanese eighth-graders (Ministry of Education, n.d.). This is 
especially true for content dealing with collecting real world data, making tables and graphs, and considering 
the nature of the resulting distributions, patterns of frequencies, and central tendencies. 

Item C20 provides students with a combinatorial question equivalent to the proverbial "handshake" 
counting problem. In this case, a student’s solution to a counting problem for games in a tournament is 
presented and the students taking the examination are asked which of four other problems can be solved 
using the same solution paths employed by the student in solving the tournament problem. The data indicate 
that 36 percent of the Japanese students were able to complete this problem correctly. 



C20. 



There are 4 baseball teams. In order to find out how many games are possible when 
every team plays with every other team, Isao thought of the following: 3 + 2 + 1 = 6, so 6 
games. Which one of the following problems can be solved using the same method that 
Isao used to solve the problem. Select one answer from (1) to (5). 

(a) Finding how many ways of numbers are possible when we make 2-digit 
numbers using 4 cards that have a number 1, 2, 3, or 4. 

(b) Finding how many ways of putting fruits into a basket are possible when we 
choose 2 fruits out of 4 different fruits. 

(c) Finding how many ways of lining up are possible, when 4 children line up 
vertically. 



(d) Based on the 4 points in the 
figure shown, find how many 
lines we can draw when we 
draw lines which go through 
2 points. 




a) & (c) 

a) & (d) 

b) & (c) 

b) & (d) 

c) & (d) 




1) 13.0 



2) 15.9 3) 12.9 4) * 36.0 5) 16.6 (no answer) 5.5 
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An analysis of problems on the grade eight portion of the 1992 NAEP located one secure item testing 
similar material. This item was one of the extended student constructed response items. It required students 
to consider various combinations possible when two coins are split into two groupings. In particular, the item 
extended the situation to ask for the probability that the student described would have sufficient money in 
one pocket to make a particular type of purchase. Data shows that 18 percent of U. S. eighth-graders were 
able to give a satisfactory or better response to this extended response item. 

The second Data Analysis item contained in the Japanese essential mathematics assessment dealt with 
averages. Item D20 presented students with a frequency distribution for an experiment of throwing a ball for 
distance. Students are asked to select a statement about the mean of one group, given some information about 
the mean of the other group of students taking part in the experiment. Students should be able to solve the 
problem by considering the relative positions of data points in the two distributions. Fifty-three percent of 
the students in the Japanese classrooms responding to this item selected the correct answer. 

D20. The table below shows the frequency distribution of the results of throwing a ball for 

8th-grade female students in the classes A and B. The average for class A is 13.3m. 
From the table, what fact can you find about the average of class B? Select one answer 

(1) to (5). 



Distance of throwing ball 


Number of students 


(m) 




Class A 


Class B 


more than 20 


less than 22 


0 


2 


more than 1 8 


less than 20 


1 


0 


more than 1 6 


less than 1 8 


3 


4 


more than 14 


less than 1 6 


2 


5 


more than 12 


less than 1 4 


8 


5 


more than 10 


less than 1 2 


4 


1 


more than 8 


less than 1 0 


2 


2 


Total 


20 


19 



(1) The average of class B is smaller than the average of class A. 

(2) The average of class B is equal to the average of class A. 

(3) The average of class B is bigger than the average of class A. 

(4) The average of class B is 2 m smaller than the average of class A. 

(5) We cannot say either one of the classes is bigger than the other class. 

1) 20.9 2) 4.9 3) *53.3 4) 5.8 5) 13.9 (no answer) 1 .2 



By contrast, U. S. students were asked, in a secure item from the 1992 assessment, to give the mean 
examination score for a small group of students whose test scores were presented in a frequency diagram. 
Only 16 percent of the U. S. students were able to solve this item requiring the use of a weighted average, 
an item that is directly related to curricular objectives at the grade eight level. 

The third item from this content category was a probability item (D21) that sampled students’ 
knowledge about long-run behavior of outcomes in an experiment based on drawing an object, with 
replacement, from a known distribution of objects. Only 27 percent of the Japanese students were able to 
correctly answer this item. 
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D21. 



There are two red balls and eight blue balls in a bag. From this bag, take one ball at a 
time and examine the color of the ball and then return it to the bag. When we repeat this, 
which one of the following is right? Select one answer from (1) to (5). 

fl) When you take a ball only once, the ball is blue. 

(2) If the first ball is blue, the second ball will be red. 

(3) If we repeat this 10 times, 8 times you will certainly get blue balls. 

(4) If we repeat this 100 times, about 80 times, we may get blue balls. 

(5) No matter how many times we repeat this, we cannot predict how many times 
we may get blue balls. 

1) 3.5 2) 2.9 3) 12.6 4) *27.2 5) 52.6 (no answer) 1.1 



There was one somewhat related probability item in the secure 1990 NAEP items for grade eight. It 
dealt, however, with sampling objects from a large set to estimate the number of defective objects in the set. 
Thirty-six percent of the U. S. students were able to correctly answer this item. 

Algebra and Functions 

The content area of Algebra and Functions contains nearly half of the items presented to students on 
the Japanese tests. Algebra and the study of variation in the form of direct proportion is at the heart of the 
grade one and grade two (U.S. eighth-grade equivalent) mathematics curricula for lower secondary schools 
in Japan. This emphasis is strongly represented in the items chosen for this portion of the essential 
mathematics assessment. 

Item D1 6 requires students to abstract a pattern from an arrangement of matchsticks and evaluate their 
pattern for the case of 20 squares. Students performed well on this item with 69 percent of them correctly 
answering it. 



D16. 



Like the figure shown, we made squares with matches. When we lined up squares and 
made a rectangle, we used 7 matches to make 2 squares and 10 matches to make 3 
squares. How many matches do we need to make 20 squares? Select one answer from 

(1) to (5). 



(1) 60 matches 

(2) 61 matches 

(3) 70 matches 

(4) 79 matches 

(5) 80 matches 



1) 10.4 2) *68.6 3) 14.2 4) 3.4 5) 3.2 (no answer) 0.3 



A similar item on the 1 990 NAEP examination, set in a similar but slightly more abstract context, asked 
for the evaluation of the pattern for a larger case value. The resulting achievement level was 32 percent 
correct. 
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NAEP 32. If this pattern of dot-figures is continued, how many dots will be in the 1 00th figure? 



3 4 

(A) 100 

(B) 101 

(C) 199 

(D) 200 

(E) 201 

A) 6 B) 12 C) 17 D) 29 E) *34 omit) 2 

A similar pattern-based problem C9 asked students to evaluate the meaning of an expression, containing 
a variable, that was used to describe the cT figure in the pattern. This item proved more difficult than just 
describing the pattern and its values as in the previous problem D16. Only 45 percent of the students were 
able to note that (a - 1) represents the number of horizontal, right-headed matches. 




C9. 



We make the following figures with matches. We found how many matches were 
needed to make ath figure from the following equation. 



3 x a + (a - 1) 

In the above equation, what does (a - 1) express? 



(1) Number of matches which are 

(2) Number of matches which are 

(3) Number of matches which are 

(4) Number of triangles which are 

(5) Number of triangles which are 



A 

A 

V 



1) *45.1 2) 10.7 3) 20.6 4) 16.5 5) 5.9 



W\ 



(no answer) 1.2 



This problem has a similar counterpart in one of the extended student constructed response items from 
the 1992 NAEP. This item, shown below, asked students to formulate an expression describing the 
relationship between the case number for the pattern and the number of dots involved and then evaluate it 
for the 20™ step in the process. This is a slightly harder item than the corresponding C9. Data from the 
assessment showed that 62 percent of the students answered the item incorrectly, 9 percent gave a minimal 
response, 6 percent a partial response, 1 percent a satisfactory response, and 4 percent an extended response. 
Thus only 5 percent of U. S. eighth-graders could give a satisfactory or better response to this item (Dossey, 
et al., 1993). 




27 



31 



NAEP 9. A pattern of dots is shown below. At each step, more dots are added to the 
pattern. The number of dots added at each step is more than the number 
added in the previous step. The pattern continues infinitely. 



(1st step) (2nd step) 



• • 

2 Dots 



• • • 
• • • 

6 Dots 



(3rd step) 

• • • • 
• • • • 
• • • • 

12 Dots 



Marcy has to determine the number of dots in the 20th step, but she does not 
want to draw all 20 pictures and then count the dots. 

Explain or show how she could do this and give the answer that Marcy 
should get for the number of dots. 



Another item type which assesses student understanding of variation is one which presents a graphical 
representation and then asks students to interpret the relation shown between the two variables involved. In 
item D1 7, Japanese students had to carefully examine the graphs relating the distances people are from point 
P at time*. They were then asked to find which of the five descriptions provided was incorrect. Sixty percent 
of the Japanese eighth-grade students were able to correctly note that neither person changed their speed for 
the times that they were in motion. Some might contend that the incorrect response (1) which the students 
were asked to find could have been a bit more explicit about this. This type of item had no counterpart in 
either the 1990 or 1992 NAEP examinations, but an item of this type is planned for the next NAEP 
mathematics assessment. Several of the newest American textbooks are including items of this type in 
teaching about functional relationships and their representations. 



D17. 



The graph below shows the traces by which two people, A and B, walked a street that 
connects point P and point Q. It indicates a relationship between time and location of 
the two people, time on * axis and distance from point P on y axis. Which of the 
following is not right among the information we get from this graph? Select one answer 
from (1) to (5). 



(1) one person changed his speed on his 
way. 

(2) one person took a rest on his way. 

(3) two people did not start at same time. 

(4) two people met each other on their way. 

(5) two people arrived at same time. 




1) *60.3 



2) 12.6 3)10.0 4) 4.8 



5) 11.5 



(no answer) 0.8 



Item Cl 5 could be classified in either the geometry or algebraic content area. It has no direct counterpart 
in the NAEP items and is difficult to place in the NAEP content categories. Its closest description is that of 
an item that requires students to translate geometric and numerical representations into symbolic form. Part 
of the translation required that the students represent the relationship between the length of segment MN and 
the lengths of segments AD and BC. For that reason, it was classified in the Algebra and Functions category. 
This item was relatively easy for students to complete correctly, as 62 percent of the students answered it 
correctly. 

Cl 5. Read the following sentences. “In a trapezoid ABCD in which AD and BC are parallel, 

the midpoints of AB and CD are Mand N. In this case, the line segment MN is parallel to 
the line segment BC and the length of MN is equal to half the length of the sum of the 
length of AD and BC.” 




(1) MV | |.BC 

(2) MN=(AD + BC)/2 

(3) MN\\BC, MN = (AD + BC) / 2 

(4) DN=NC, MN= (AD + BC) /2 

(5) None of the above 

1) 5.5 2)15.1 3) * 61.8 4) 8.7 5) 6.9 (no answer) 2.1 



Item C4 required students to translate a problem that was described verbally into mathematical terms 
and solve it in terms of a specified time. Then the problem was shifted to a slightly more abstract form and 
a related question about the round-trip time was asked, which required a symbolic expression for an answer. 
The shift from the first part, where 67 percent of the students gave the correct response, to the second part, 
where 59 percent of the students gave the correct response, shows the decrease in performance that might be 
attributed to the shift from concrete to abstract settings. 

C4. We make a round trip to the top of a mountain. It is 6km from here to the top of the 

mountain. 




(A) Go by foot with a speed of 3 km per hour, take a rest for 30 minutes at the top 
of the mountain, and return by foot with a speed of 4km per hour. How many 
hours does it take for a round trip. Select one answer from (1) to (5). 

(1) 3 hours and 30 minutes 

(2) 4 hours 

(3) 5 hours 30 minutes 

(4) 6 hours 

(5) 7 hours 30 minutes 



1) 13.2 



2) * 67.0 3) 8.5 4) 4.9 5) 4.9 (no answer) 1.4 



(B) Go by foot with a speed of a km per hour, rest 30 minutes at the top of the 
mountain, and return by foot, witn a speed of 6 km per hour. How many 
hours does it take for a round trip. Select one answer from (1) to (5). 

(1) 3a + 46 +1/2 

(2) 6a + 66 + 1/2 

(3) ab + 1/2 

(4) 6/a + 6/6 +1/2 

(5) a/6 + 6/6+1/2 

1) 8.8 2) 6.7 3) 5.6 4) * 58.9 5) 18.3 (no answer) 1.6 
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There are no comparable NAEP items at the grade eight level. A similar item was field tested in 1993, 
but so few students were able to perform on it that it was omitted from the test form in favor of an item that 
would at least be able to discern differences in student performance. 

Item C19 measured students’ ability to select an ordinate for point C so that the points A, B, and C all 
fall on a straight line. Students could approach this by considering the relationship existing between the 
abacuses and ordinates in the other two ordered pairs given, or simply graph the first two and extrapolate the 
pattern to reach point C, noting its y-value. Given the ease of answering this item, only 31 percent of the 
students were able to correctly select the answer to this item. There was no directly comparable item in the 
NAEP items that have been administered. 

Cl 9. Enter a number into the [ ] so that the 3 points A, B, and C are on one line. 

A (2,1), B (5,7), C (18, [ ]) 

Select one answer from (1) to (5). 

(1 
(2 
(3 
(4 
(5 

1) 16.2 2)16.2 3)22.3 4) *30.9 5) 10.7 (no answer) 3.6 

Item D1 was a straightforward item dealing with the location of points and their coordinates on an 
integer number line. This item had the highest percent of students responding correctly, 94 percent. There is 
no similar item on the present NAEP examinations, but one is planned for the next assessment. 

D 1 . Find a point which is larger than -3 by 2 on the line below. Select one answer from 

(1) to (5). 



] 15 
) 27 
) 33 
) 36 



1 < l > 1 l— <6 <b 1 1 1 <t> 1 1 <k> 1 

1) 3.5 2)1.1 3) *93. 8 4) 0.7 5) 0.6 (no answer) 0.3 



The next item was one of the items that was used to consider the equivalence of the two Japanese forms 
given. Item C5, which required students to select an equation that represents the problem described, also 
appeared as item D6. Student performance data on this item showed 70 percent of the students taking Form 
C getting the item correct, while 67 percent of those taking Form D got the item correct. There again is no 
directly comparable NAEP item, although a similar item is planned for the 1996 assessment. 



C5/D6. 




Mr. A planned to read a book within a certain number of days. At first he planned to read 
30 pages a day; he noticed that 50 pages would be left at the end of the period. Then he 
changed his plan and decided to read 35 pages a day. However, 1 5 pages would still be 
left at the end of the period. In order to know the total number of days and total number 
of pages of the book which Mr. A is trying to read, we set up an equation, where x is the 
total number of days Mr. A would have to spend to read tne book. Select one answer 
from (1) to (5). 

(1) 30(x + 50) = 35(x + 15) 

(2) 30* + 50 = 35* + 15 

(3) 30* - 50 = 35* — 15 

(4) 30 + x + 50 = 35 + x + 15 



(5) 30/x + 50 = 35/x + 15 














FORM C: 


1) 9.5 


2) *69.5 


3) 


10.0 


4) 


4.5 


5) 5.1 


(no answer) 1 .4 


FORM D: 


1) 9.0 


2) *66.6 


3) 


10.9 

30 


4) 


4.8 


5) 8.1 


(no answer) 0.6 
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Item D4 asks students to select an equation that describes a physical situation involving the arranging 
of marbles. The situation is the classic context where students have to take care not to double count marbles 
in the comers of the arrangement. Both ploys ( 1 ) and (4), which contain this error explicitly, with no attempts 
to adjust, account for 34 percent of the responses. Forty-eight percent of the students answered the question 
correctly. No NAEP item directly measures the same content as measured by item D4. 

D4. We lined up a pieces of marbles vertically and b pieces of marbles horizontally. Which 

one of the following equations expresses the total number of marbles? Select one 
answer from (1) to (5). 




(1) 2 (a + b) 

(2) 2 (a + o) - 4 

(3) 2(a + b)-2 

(4) ab 

(5) ab-{a-\)(b-\) 

1) 22.7 2) *48.0 3) 13.7 4) 11.6 5) 3.6 (no answer) 0.3 



Item D9 shifts the interest to the solution of inequalities, as it asks the Japanese youth to find the 
inequality equivalent to x/2<7. This was a relatively easy item for students, as 68 percent of the students got 
it correct. 



D9. Solve the following inequality and select one answer from (1) to (5). 

x/2 < 7 

(1) x > 5 

(2) x> 14 

(3) x < 7/2 

(4) x < 5 

(5) x < 14 

1) 2.0 2) 9.8 3) 17.3 4) 2.5 5) *68.0 (no answer) 0.4 



Two similar items appeared on the grade eight portions of NAEP examinations. In 1990 students were 
asked to select the least whole number satisfying 2x>l 1 and in 1992 to supply two whole numbers that would 
satisfy the open sentence 54 < 3 xO. Performance on these two items was 45 percent correct and 49 percent 
correct respectively. This is somewhat disappointing in that the Japanese item is harder than either of the 
corresponding NAEP items as it asks students to make a transformation of the inequality and then identify 
an equivalent inequality. 

NAEP 25. What is the least whole number x for which 2x > 1 1? 

(A) 5 

(B) 6 

(C) 9 

(D) 22 

(E) 23 

A) 5 B) *46 C) 10 D) 20 E) 1 omit) 1 
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NAEP26. Write two numbers that could be put in the □ to make the number sentence below true. 
54 <3 xD 



Item Cl tests the order of operations. Many students make the error of adding first, obtaining a 13 and 
then finding the product of 13 and -5. That this error pattern is universal is shown by the 1 1 percent of the 
students selecting the incorrect answer of -65. As this item is a target of instruction, and apparently learned, 
at this grade level, 85 percent of the students answered it correctly. 



Cl. 



Compute the following. 

9 + (+4) x (-5) 

Select one answer from (1) to (5). 




8 

-9 

-11 

-25 

-65 



1) 1.8 2) 0.4 3) *84.5 4) 1.2 



5) 10.9 (no answer) 1.2 



Two NAEP items also measure students’ knowledge of order of operations. The first item, a secure 
item in 1990, is very similar to Cl, but it doesn’t involve multiplication by a negative number. Ninety-four 
percent of U. S. grade eight students answered this item correctly. The second item from 1992, illustrated 
below, combines five operations, counting the exponentiation. This level of complexity brought performance 
of eighth-graders in the U.S. down to 22 percent correct. 

NAEP 3. 3 3 + 4(8 — 5) — S- 6 = 

(A) 6.5 

(B) 11 

(C) 27.5 

(D) 29 

(E) 34.16 

A) 26 B) 10 C) 15 D) *22 E) 22 omit) 5 



Question C2 asked students to select which one of a number of statements about operations with integers 
was not always true. The first statement that says that a positive number plus a negative number is always a 
positive number stands out quickly as the obviously correct answer. However, it was not quite so obvious to 
Japanese students, as only 66 percent of them answered the item correctly. This is still a relatively high level 
of performance, but beneath what might be expected. There was no directly comparable NAEP item in the 
pool. 
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C 2. Which one of the following mathematics statements is not always true. Select one answer 

from (1) to (5). 

(1) (positive number) + (negative number) = (positive number) 

(2) (negative number) + (negative number) = (negative number) 

(3) (positive number) - (negative number) = (positive number) 

(4) (negative number} - (positive number) = (negative number) 

(5) (negative number) x (positive number) = (negative number) 

1) *66.3 2) 8.1 3) 8.4 4) 11.0 5) 5.1 (no answer) 1 . 1 



Item C3 required students to place an expression involving the difference of an expression and twice 
a second expression into a simpler form. This item also appeared as item D5 on the second form of the test. 
This was one of the easier items for Japanese students. Eighty-one percent of the students got it correct on 
form C and 79 percent got it correct on Form D. The remaining students basically had one of two common 
difficulties in dealing with the -2 coefficient on the second expression. This item has no direct counterpart 
on the present NAEP assessments for eighth-graders, but a similar item is planned for the next assessment. 



C3/D5. Compute the following. (4 a - 6) - 2(a - 3) 
Select one answer from (1) to (5). 

(1) 2a 

(2) 2a -3 

(3) 2a- 12 

(4) 3a -9 

(5) 3a - 1 1 



FORM C: 


1) 


*80.5 2) 


9.1 


3) 


7.8 


4) 


1.0 


5) 0.8 


(no answer) 0.8 


FORM D: 


1) 


*78.5 2) 


14.2 


3) 


4.9 


4) 


1.3 


5) 0.8 


(no answer) 0.3 



Item D19 asks students to either extrapolate a pattern given in a graph to find the altitude at which the 
temperature first reaches 0°C or to set up an equation representing the relationship between temperature and 
altitude illustrated and then solve for the altitude, given the temperature of 0°C. As shown, 66 percent of the 
students were able to correctly solve the problem posed and represented by the graph relating the two variables. 
This item is similar to items beginning to appear in U. S. algebra textbooks and teaching materials aimed at 
helping students develop a meaning of variable. However, there is no current NAEP item similar to this item. 



D 1 9. The graph below indicates a relationship between x and y, the temperature of _y°C at 

the altitude of x m. At what altitude (m), does the temperature become 0 C. Select 
one answer from (1) to (5). 



(1) 3000 m 

(2) 4000 m 

(3) 5000 m 

(4) 10000m 

(5) 26000 m 




1) 8.0 2) *66.3 3) 10.9 4) 7.0 5) 6.9 (no answer) 0.8 



ERiC 
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